Tracking health system performance in times of crisis using routine health data: lessons learned from a multicountry consortium.

COVID-19 has prompted the use of readily available administrative data to track health system performance in times of crisis and to monitor disruptions in essential healthcare services. In this commentary we describe our experience working with these data and lessons learned across countries. Since April 2020, the Quality Evidence for Health System Transformation (QuEST) network has used administrative data and routine health information systems (RHIS) to assess health system performance during COVID-19 in Chile, Ethiopia, Ghana, Haiti, Lao People's Democratic Republic, Mexico, Nepal, South Africa, Republic of Korea and Thailand. We compiled a large set of indicators related to common health conditions for the purpose of multicountry comparisons. The study compiled 73 indicators. A total of 43% of the indicators compiled pertained to reproductive, maternal, newborn and child health (RMNCH). Only 12% of the indicators were related to hypertension, diabetes or cancer care. We also found few indicators related to mental health services and outcomes within these data systems. Moreover, 72% of the indicators compiled were related to volume of services delivered, 18% to health outcomes and only 10% to the quality of processes of care. While several datasets were complete or near-complete censuses of all health facilities in the country, others excluded some facility types or population groups. In some countries, RHIS did not capture services delivered through non-visit or nonconventional care during COVID-19, such as telemedicine. We propose the following recommendations to improve the analysis of administrative and RHIS data to track health system performance in times of crisis: ensure the scope of health conditions covered is aligned with the burden of disease, increase the number of indicators related to quality of care and health outcomes; incorporate data on nonconventional care such as telehealth; continue improving data quality and expand reporting from private sector facilities; move towards collecting patient-level data through electronic health records to facilitate quality-of-care assessment and equity analyses; implement more resilient and standardized health information technologies; reduce delays and loosen restrictions for researchers to access the data; complement routine data with patient-reported data; and employ mixed methods to better understand the underlying causes of service disruptions.

The unintended consequences of COVID-19 mitigation measures matter: practical guidance for investigating them.

BACKGROUND: COVID-19 has led to the adoption of unprecedented mitigation measures which could trigger many unintended consequences. These unintended consequences can be far-reaching and just as important as the intended ones. The World Health Organization identified the assessment of unintended consequences of COVID-19 mitigation measures as a top priority. Thus far, however, their systematic assessment has been neglected due to the inattention of researchers as well as the lack of training and practical tools. MAIN TEXT: Over six years our team has gained extensive experience conducting research on the unintended consequences of complex health interventions. Through a reflexive process, we developed insights that can be useful for researchers in this area. Our analysis is based on key literature and lessons learned reflexively in conducting multi-site and multi-method studies on unintended consequences. Here we present practical guidance for researchers wishing to assess the unintended consequences of COVID-19 mitigation measures. To ensure resource allocation, protocols should include research questions regarding unintended consequences at the outset. Social science theories and frameworks are available to help assess unintended consequences. To determine which changes are unintended, researchers must first understand the intervention theory. To facilitate data collection, researchers can begin by forecasting potential unintended consequences through literature reviews and discussions with stakeholders. Including desirable and neutral unintended consequences in the scope of study can help minimize the negative bias reported in the literature. Exploratory methods can be powerful tools to capture data on the unintended consequences that were unforeseen by researchers. We recommend researchers cast a wide net by inquiring about different aspects of the mitigation measures. Some unintended consequences may only be observable in subsequent years, so longitudinal approaches may be useful. An equity lens is necessary to assess how mitigation measures may unintentionally increase disparities. Finally, stakeholders can help validate the classification of consequences as intended or unintended. CONCLUSION: Studying the unintended consequences of COVID-19 mitigation measures is not only possible but also necessary to assess their overall value. The practical guidance presented will help program planners and evaluators gain a more comprehensive understanding of unintended consequences to refine mitigation measures.